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ABSTRACT 

"^^i.ghty first-grade students^ randomly selected from 
eight elementary Echools^ participated in this study. Four of the 
schools were located within the city and four in the rural area 
surrounding the city. The assessment instrument was Ihe Organisms 
Evaluation Sup t, designed to evaluate students' attitudes in 

science^ students' perception of the classroom environment, and how 
well students are achieving the specified cognitive objectives of the 
unit. The instrument was administered as a pretest and a posttest to 
all students. The treatment consisted of instructing the students in 
the "Organisms" unit of the Science Curriculum Improvement Study 
(SCIS) program. Significant positive change in pupil performance as a 
result of instruction was found. The gain scores were subjected to a 
t-test at the ^05 level. No pretest or posttest differences were 
found when examining urban versus rural populations. It was concluded 
that the "Organisms" assessment instrument is effective in terms of 
content validity, reliability, objectivity and interest. 
(Author/EB) 
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introduction 

The Science Curriculum Improvement Study elementary science 
program is one of several curriculum development projects which were 
funded by the National Science Foundation. The program is organized 
around a conceptual framework which is bound to^'ether by the processes 
of science. The main objective of the program is to produce students 
who are scientifically literate. 

Most of the evaluation of the project which has been done has 
been to insure proper construction of the units. In 1970, an evalua- 
tion device was designed by the developers that would enable classroom 
teachers to evaluate student progress in the program. At the time of 
this .study no data was available on the effectiveness of this classroom 
assessment instrument. 

Statement of the Problem 

What is the effectiveness of the Concept/ Process section of the 
assessment instrument for the Science Curriculum Improvement Study 
first grade biological science unit entitled "Organisms^*? 

Assumptions and Limitations of the Study 

This study was delimited to eight first grade classrooms in 
the Chambersburg Area School District, Chambersburg;» Pennsylvania. 
The researcher was not the regular classroom teacher of the students, 
therefore the students may have been reluctant to respond to the 
questions of the researcher. 
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students were removed from their classrooms for individual 
testing. This may have created a stross situation which could have 
affected their responses* 

Delimitations of the Study 

The researcher assessed only the Concept/Process section of 
the S. C. I. S. instrument in terms of content validity and reliability. 
The attitudes in science and the perception of classroom erivironmetit 
sections of the assessment instrument were not included in this study. 

It was assumed by the researcher that because of the student 
oriented nature of the Science Curriculum Improvement Study program, 
teacher differences would not be a threat to the validity of the study. 

H ypotheses 

1. The Concept/Process section of the assessment instrument 
for the "Organisms" unit of the Science Curriculum Improvement Study 
program is effective in terms of content validity and reliability. 

2. There is a positive change in pupil performance between 
a pre-test and a post-test of the Concept/ProcGrss section of the 
"Organisms" assessment instrument. 

3. There is no significant difference in pupil performance 

on the Concept/Process section of the "Organisms" assessment instrument 
between pupila in rural schools and those in urban schools. 

Sample 

The sample was limited to first grade students in the Chambersburg 
Area School District who were receiving instruction in the S. C. I. S. 
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elementary science program. Ten students, five girls and five boys, 
wer,e selected at random frora the first grade classes of each of eight 
elementary schools in the district. The random selection of subjects 
for the sample groups w£s done by the researcher from class lists 
supplied by the teachers, A total of eighty first grade students 
were chosen for participation in the study. 

The schools were selected on the basis of geographical location. 
Four of the cooperating elementary schools were located within the city 
of Chambersburg, a city of approximately 25,000 people located in the 
rural Cumberland Valley of Pennsylvania. Four were schools in the 
rural area surrounding Chambersburg. 

Design of the Instrument 

The S. C. I. S. first grade biological science unit is entitled 
"Organisms.'^ The assessment instrument for this unit was prepared for 
C. I. S. by Knott, Lawson, and Peterf^on.^ The instrument is divided 
into three areas of pupil assessment. 

The first area is designed to evaluate students^ attitudes in 
science. Four desired attitudes are examined^ These attitudes are: 
curiosity, inventiveness, critical thinking, and persistence. 

The second. section of the assessment instrument is designed to 
evaluate the students^ perception of the classroom environment. Pupils 
are given an opportunity to express their feelings about the science 
activities they have completed. 

The third section of the instrument is the Concept/Process 
evaluation section. This section consists of five activities which 
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seek to evaluate how well students are achieving the specified cogni- 
tive objectives of the ^^Organisms" unit. Activity one, "Identifying 
Requirements for Plant Growth," assesses the children's abilities to 
identify and describe requirements necessary for seed germination and 
plant growth. Activity two, "Describing Events in Aquaria," assesses 
the children's abilities to describe events that occur in aquaria: 
feeding, birth, hatching, growth, death, and decay. Activity three, 
"Identifying Organisms and Habitats," assesses the children's abilities 
to identify pictures of organisms and habitats and to associate specific 
organisms with specific habitats. Activity four, "Diagramming Food 
Webs," assesses the children's abilities to construct food webs. 
Activity five, "Discussing Experiments," assesses the children's 
abilities to pose hypotheses, suggest experiments for testing hypo- 
theses, and drawing conclusions from experimental data."^ 

Activites one, two, and five of the Concept/Process section 
of the instrument .vere administered to members of the sample groups 
as individual tests while activities three and four were given as 
group tests. 

Procedures fo r Gathering Data 

The Concept/Process section of the organisms assessment 
instrument was administered by the researcher as a pre-test and a 
post-test to all students included in the study. Only one evaluative 
activity was administered to the students in a given sitting. Teachers 
who were participating in the study agreed to follow the schedule 
outlined in the Organisms Teachers Guide. Pretests were administered 
to the sample groups during the weeks January 15 to January 31. 
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At the conclusion of the pre-test the teachers began instructing 
the students in rhe '^Organisms unit of the S. C. I. S. program. As 
each section of the "Organisms^^ unit was completed, teachers contacted 
the researcher and a time was arranged to administer the corresponding 
section of the post-test. 

The pre-test scores of students in the rural and urban groups 
were subjected to a t test which indicated that there were no differences 
in the beginning operational levels of the groups before they were 
subjected to the treatment. 

The treatment consisted of instructing the students in the 
"Organisms" unit of the S. C. I. S. program. The pre-test and post- 
tests were administered in the same manner. 

The pre- and post-test scores of both th^ rural and urban 
groups were then compared to determine if a positive change in pupil 
performance had occurred. Gain scores were calculated for the rural 
and urban groups and a t test of difference was applied to the scores. 

Reliability for the post-test scores Concept/ Process section 
of the assessment instrument was determined by the split half method 
using the Spearman-Brown formula. 

Content validity was established by comparing the Concept/ 
Process objectives in the "Organisms" teachers manual to the test 
items of the Concept/Process section of the assessment instrument. 
This was done by two first grade teachers who had taught the Organisms 
unit, one Science educator, and the researcher. 

Post-test scores for students in rural and urgan groups were 
subjected to a t test of difference to determine if there was a 

ERIC 



6 



significant difference between rural and urban pupil performance on 
the post-test. 

Method of Analysis of Data 

The pupil responses for both pre-test and post-tests were 
tabulated. The items were listed on pre-test and post-test master 
sheets for each school according to the order in which they appear on 
the evaluation instrument. The items were numbered from one to 
forty-one. A correct response for each item was given the value of 
one point. 

The number of students participating in the study was eighty. 
Pre-test data were based upon this number. However, during the course 
of the study three mortalities resulted in the rural population. 
Therefore post-test scores and gain scores were computed on the basis 
of a population of seventy-seven pupils. 

The pre-test scores for pupils in rural and urban schools were 
subjected to a t test of difference. Calculation showed that the urban 
mean for the pre-test was 11.4 with a standard deviation of 2.57 and 
the rural mean for the pre-test was 11.6 with a standard deviation of 
3.10. (See Table 1.) In a comparison of the rural and urban pre-test 
scores the calculated value of t at the .05 alpha level was 0.353 with 
78 degrees of freedom. This value was smaller than the necessary 2,000^ 
value as taken from statistical tables. Therefore, one can infer that 
no significant difference exists between the pre-test scores of rural 
and urban students. It was assumed, therefore, that students were at 



Table 1 



Pre~Test and Post-Test Mean Scores 
and Mean Gain Scores for all 
Schools in the Population 











Pre-Test 




Post-Test 


Mean 




School 


N 


Mean 


N 


Mean 


Gain 




URBAN 








. 




I. 


School A 














Boys 


5 


10.4 


5 


26.8 


16.4 




Girls 


5 


10.4 


5 


27.8 


17.4 




Total 


IG 


10.4 


10 


27.3 


16.2 


II, 


School B 














Boys 


5 


13.0 


5 


25.8 


12.8 




Girls 


5 


11.0 


5 


24.8 


12.2 




Total 


10 


12.0 


10 


25.3 


IS. 3 


III. 


School C 














Boys 


5 


12.6 


5 


27.4 


14.8 




Girls 


5 


14.0 


5 


28.8 


14o8 




Total 


10 


13.3 


10 


28.1 


14.8 


IV. 


School D 














Boys 


5 


10.8 


5 


28.6 


17.8 




Girls 


5 


9.0 


5 


28.2 


19.2 




Total 


10 


9.9 


10 ■ 


28.4 


18.5 


URBAN TOTAL 


40 


11.4 


40 


27.2 


15.8 



NOTE: Table 1 continued on next page 



Table 1 (continued) 









rre-iesr 




rost-iest 


Mean 




School 


N 


Mean 


N 


Mean 


Gain 




RURAL 












T 

i • 
















Boys 


5 


12.8 


5 


32.0 


19.2 




Girls 




12,4 


5 


27.4 


T r* 0 

15.0 




Total 


10 


12.6 


T r» 
10 


on n 

2.9 . 7 


T ^7 T 

17 .1 


T T 
1.1. 


Scnool F 




=. 










Boys 


5 


8.4 


5 


26.8 


18.4 




Girls 


5 


10.0 


4 


o o r> 

28 . 0 


TO 0 

lo . 2 




iO l31 




V . 1 


y 




1 Q A 

lo . 4 


T T T 

ill. 


Sciiool G 














Boys 


5 


12.2 


5 


26.4 


14.2 




Girls 


5 


11.8 




25.2 


13.4 




Total 


10 


TO /T 

12.0 


10 


0 c 0 

25 . 0 


To 0 

13 . 8 


IV. 


School H 














Boys 


5 


11,4 


o ■ 


22.3 


11.0 




Girls 


5 


14.4 


5 


27.6 


13.2 




Total 


10 


12.9 


8 


25.6 


12.3 


RURAL 


TOTAL 


40 


11.6 


37 


27.1 ■ 


15.4 




TOTAL 


POPULATION 


80 


11.5 


77 


27.2 


15.6 



o 
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approximately the same level of performance on the pre--test items 
prior to being given instruction in the '^Organisms'^ \init. 

Analysis of Data 

Hypothesis number one: The Concept/Process section of the 
assessment instrument for the "Organisms" unit of the Science Curri- 
culum Improvement Study is effective in terms of content validity and 
reliability. 

In order to determine the content validity of the instrument, 
copies of the Concept/Process section of the assessment instrument and 
a copy of the "Organisms" Teacher^ s Guide were supplied to two first 
grade teachers in the Chambersburg School District. These teachers 
had taught the C. I. S. "Organisms" unit. In addition, a copy of 
rhe assessment instrument and an "Organisms" Teacher^ s Guide were 
inspected by a science educator and the researcher. These four indivi- 
duals compared the specific objectives within the teacher's guide with 
the items on the evaluation instrument. A consensus of the four opinions 
was used to derive a single statement of content validity by the 
researcher. That statement is as follows: t\\e items of the Concept/ 
Process section of the assessment instrument for the "Organisms" unit 
of the S. C. I. S. program are consistent with the objectives of the 
Organisms Teacher's Guide. The instrument, therefore, is considered to 
be valid. 

To determine the reliability of the Concept/Process section of 
the "Organisms" assessment vnstrument, only the scores of the post-test 
were used. The test itfms were si>lit into two groups, A and B. The 
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test itens were split into t\^o groups, A and B. The items were paired 
according to equivalent difficulty, as determined by the researcher. 
Each test paper was then assigned two groups of scores. A Pearson 
product-moment correlation coefficient was computed for the two sets 
of scores. This was found to be .405. The Pearson product-moment 
correlation obtained was equivalent to a test one half the length of 
the test used. 

Using the Spearmen-Brown formula, an r of .577 was obtained. 
This is considered to be significant. The Concept/Process section of 
the "Organisms assessment instrument is therefore considered to be 
a reliable test. 

Since the '^Organisms'^ assessment instrument was found to be 
both valid and reliable, the first hypothesis of this study must be 
accepted. 

Hypothesis number two: There is a positive change in pupil 
performance on the Concept/Process section of the ^'Organisms" assess- 
ment instrument from pretest to post-test. 

To test this hypothesis gain scores were computed for each 
student. The total mean gain for the population of seventy-seven 
students is 15.4. The results are shown in Table 1. It is apparent 
that the gain scores are significant when a t test is applied. At 
the .05 alpha level with 76 degrees of freedom, a calculated value 
for t was found to be 35.1. Since this is more than the 1.67 needed 
to prove significance, the positive change in pupil performance is 
considered to be highly significant. Thus the second hypothesis of 
this study cannot be rejected. 
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Hypothesis number three: There is no significant difference 
in pupil performance on the Concept/Process section of the "Organisms^' 
assessment instrument between pupils in rural and urban schools. 

To test this hypothesis, the post-test scores of the students 
in rural and urban schools were subjected to a t test of difference. 
(See Table 1 for post-test scores.) Statistical analysis of these 
scores indicates the calculated t value at the .OS alpha level with 75 
degrees of freedom as being 0.23. Since this is smaller than the 
2.000 level necessary to establish a significant difference, the 
hypothesis cannot be rejected. Therefore, the researcher concluded 
there was no significant difference between rural and urban pupil 
performance on the post-t€'5t:. 

Conclusions 

The Concept/Process section of the assessment instrument of 
the "Organisms" unit of the S. C. I. S. program was found to be both 
valid and reliable. It can be considered to be an acceptable method of 
evaluating the degree to which pupils are attaining the specific 
objectives of the "Organisms" unit of the S.. C. I. S. program. 

Since there was a significant improvement in pupil performance, 
it can be inferred that the students within the population had obtained 
some of the specific goals established in the "Organisms" unit. 
Inspection of the data sheets revealed that students were able to 
describe requirements for plant growth, describe events in aquaria, 
identify organisms and habitats, associate specific organisms with 
specific habitats, and construct food webs. Forty- four out of the 
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total population of seventy-seven were cible to suggest an acceptable 
hypothesis for a problem on the post-test. However, only sixteen out 
of the seventy-seven were able to interpret the results of an experi- 
ment. No child in the population was able to suggest an experiment to 
test his hypothesis. 

In addition, children appeared to have difficulty in suggesting 
an approximate time which was required for the germination of seeds. 
Children also had difficulty in recognizing seeds as organisms. Only 
seventeen out of the population of seventy-seven were able to do this. 

The food web also presented problems for students. Children 
had difficulty in identifying the correct feeding relationships. 
Forty-seven out of the seventy-seven students correctly identified the 
patrern. Many students had difficulty using arrows to identify the 
feeding relationships. However, fifty-two students did use the arrows 
correctly. 

Implications 

From the results of the study the researcher concluded that the 
the ^'Organisms" assessment instrument is an effective test in terms of 
content validity, reliability, objectivity, and interest. The students 
observed by the researcher retained a high degree in interest throughout 
the testing program. 

Because of the activity oriented nature of the instrument, the 
students did not seem to regard it as a test. They responded in much 
the same way one would expect them to during a show and tell experience. 
Teachers within each school informed the researcher that students 
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looked forward to his vi.qit. 

The first four activities can be considered objective in 
nature. Either a student displayed an observable behavior or he did not. 
The evaluator has only to inake a yes or no decision. However, the 
fifth activity requires subjective judgment on the part of the evaluator. 
A decision must be aiade as to whether or not the student has suggested 
an acceptable hypothesis, interpreted data satisfactorily, and designed 
an appropriate experiment. These interpretations may vary with the 
standards of the evaluator.. 

The study indicated that there was no significant difference 
between the post-test scores of students in the urban and rural schools 
of the population. Chambersburg is located in a rural setting, the 
Cumberland Valley of Pennsylvania. It is possible that students in 
both urban and rural schools had similar backgrounds u For this reason, 
it is difficult to generalize this data beyond the limits of the popu- 
lation. More research is needed to determine the effect of the 
"Organisms" unit on students of greatly divergent backgrounds. 

It was noted in the conclusion that students had difficulty with 
several items on the test. It was noted that students had difficulty 
in hypothesizing, interpreting data, and constructing experiments. All 
three require advanced levels of abstraction. Most first grade students 
are at a concrete level of operation. Perhaps it is asking too much 
for these abilities to be developed after a one-year exposure to the 
S. C. I. S. program. As stated in the overview, these are "long range 
goals of the S. C. I. S. program."^ Students should be expected to begin 
to exhibit these abilities the fourth year of their involvement in the 
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program. Further testing is needed to determine when these abilities 
become apparent. 

Children also failed to respond adequately to the item dealing 
v;irh the time needed for seeds to germinate. Children of this age 
group are still formulating their concept of time. It was apparent from 
responses, which varied from seven weeks to two months, that not all 
students in the study had yet developed a concept of time. 

The items dealing with food webs also presented problems for 
many students. These items required students to transfer information 
to a new situation. Again, this is perhaps an activity which requires 
abstraction beyond the level of first grade students. 

One of the basic assumptions of this study was that the 
materials in the "Organisms" unit were child centered. Therefore, 
teacher differences were not considered to be important. This assump- 
tion is in question. The researcher supplied four sets of guppies to 
one of the schools participating in the study. Each time the guppies 
died. An investigution into the procedure used by the teacher to set up 
the aquaria revealed that she had washed the tanks with detergent. 
The resulting soap residue had killed the guppies. After the situation 
was corrected, the class proceeded without further incident. 

In a second class it was discovered that a teacher was afraid 
to give the students the responsibility of feeding the fish. As a 
result, some students were not sure what fish ate. 

The researcher also noted that the Chambersburg district had a 
problem with supplying the teachers with live materials for the "Organ- 
isms" unit. In four out of eight schools in the study, teachers were 

ERIC 



3 5 



not supplied with adequate live materials to teach the program. As a 
result, the researcher supplied live materials where needed. 

Recommenda t ions 

It is recommended by the researcher that, based upon the results 
of this study, the first grade teachers within the population should 
be supplied with the "Organisms" assessment instrument. The instrument 
would be helpful in aiding teachers to determine how well their students 
were achieving the specific objectives set forth in the Organisms 
Teacher's Guide . 

The researcher feels that portions of the assessment instrument 
need to be altered or deleted. Part five, in particular, contains test 
items which appear to be dissonant with the expected developmental level 
of first grade children. If not deleted, teachers should be warned 
about placing too high a value on these items. 

It is also recommended that a program of in-service instruction 
in science education be established for teachers within the Chambersburg 
district who are teaching the S. C. I. S. program. This would improve 
teachers^ backgrounds and techniques of instructing the program. In 
particular, teachers need information of the care of organisms. In 
addition, in-service sessions would provide an opportunity for an 
exchange of ideas among the teachers involved. 

It is a further recommendation of this study that a science 
coordinator be employed by the Chambersburg School District. The 
coordinator should be placed in charge of teacher in-service education 
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in science teaching* In addition, he should supervise the maintenance 
of the science materials which are necessary to properly operate the 
S. C. I. S. program. 



FOOTNOTES 



-'■Robert C. Knott, and others. Organisms Evaluation Supplement 
(Berkeley, California: Regents of the University of California, 1972). 

2xbid. 

^"^Robert L, Ebel, Measuring Educational Achievement (Englewood 
Cliffs, New Jersey: Prentice-Hall, Inc., 1965), 447] 

"^Sir Ronald A. Fisher and Frank Yates, Statistical Tables for 
Biological, Agricultural, and Medical R es earch (New York:' Hafner 
Publishing Company, Inc., 1963), 46. 

^Knott, op^ cit., Activity 5. 



